help measure ai model uncertainty
Meta releases Bean Machine to help measure AI model uncertainty
Let the OSS Enterprise newsletter guide your open source journey! Meta (formerly Facebook) this week announced the release of Bean Machine, a probabilistic programming system that ostensibly makes it easier to represent and learn about uncertainties in AI models. Available in early beta, Bean Machine can be used to discover unobserved properties of a model via automatic, "uncertainty-aware" learning algorithms. "[Bean Machine is] inspired from a physical device for visualizing probability distributions, a pre-computing example of a probabilistic system," the Meta researchers behind Bean Machine explained in a blog post. "We on the Bean Machine development team believe that the usability of a system forms the bedrock for its success, and we've taken care to center Bean Machine's design around a declarative philosophy within the PyTorch ecosystem." It's commonly understood that deep learning models are overconfident -- even when they make mistakes.